DIGITAL VIDEO PROCESSING METHOD AND APPARATUS THEREOF 



REFERENCE TO RELATED APPLICATION 
This application claims the benefit of U.S. provisional application Ser. 
ONo.60/1 18,750, filed Feb. 5. 1999, the disclosure of which Is incorporated herein by 
reference. 

BACKGROUND OF THE INVENTION 
1 . Field of the Invention 

The present invention relates to a digital video processing method and 
apparatus thereof, and more particularly, to a digital video processing method and 
apparatus for classifying video scenes based on the motion information of a given 
scene. 

Intensity of a video scene refers to the visual intensity experienced by a 
viewer when the viewer sees the video scene. This is an abstract concept indicating 
that the scene is very important in view of an overall video program. Thus, methods 
for searching video streams based on the intensity are required. However, as yet, 
there are no known methods for searching video streams based on the intensity nor 
methods for processing the same. 

SUMMARY OF THE INVENTION 

It is a first object of the present invention to provide a digital video processing 
method which allows for video search based on intensity. 

It is a second object of the present invention to provide a computer readable 
medium for embodying the digital video processing method. 

It is a third object of the present invention to provide a digital video processing 
apparatus for embodying the digital video processing method. 

It is a fourth object of the present Invention to provide a digital video analyzing 
method for comparing the intensiti s of video streams processed by the digital video 
processing method. 



It is a fifth object of the present invention to provide a digital video searching 
method for searching video streams based on the intensities of video streams 
processed by the digital video processing method. 

It is a sixth object of the present invention to provide a digital video search 

5 apparatus for embodying the digital video searching method. 

To achieve the first object.of the present invention, there is provided a method 
for processing digital images received in the form of compressed video streams 
comprising the step of (a) detennining a region intensity histogram (RIH) based on 
information on motion compensation of inter frames. 

10 Also, before the step (a), there may be further included the steps of (p-a) 

receiving video streams, (p-b) grouping the video streams into a plurality of groups 
using a predetermined algorithm, (p-c) selecting a group to be processed, wherein in 
the step (a), the RIH of the selected group is detennined based on information on 
motion compensation of inter frames. 

15 According to another aspect of the present invention, there is provided a 

method for processing digital images received in the form of compressed video 
streams including the step of (a) selecting an inter frame from the video streams, (b) 
with respect to the selected inter frame, calculating the ratio of the number of blocks 
whose motion compensation values are zero to the total number of macroblocks, (c) 

20 quantizing the ratio into / levels, where / is a predetermined positive number, (d) 

checking whether the processed inter frame is the last one, (e) if it is determined in 
the step (d) thatihe-processedinterframe is not the last one, selecting the next inter 
frame and performing the steps (b) through (d), (f) if it is determined in the step (d) 
that the processed inter frame is the last one, calculating the ratio of the number of 

25 inter frames in the /th level to the total number of inter frames in a given region, and 
(g) determining the ratio obtained in the step (f) as predetermined histogram 
information. 

Also, the digital video processing method may further Include the steps of (p- 
a) receiving video streams, (p-b) grouping the video streams into a plurality of 
30 groups using a predetermined algorithm, (p-c) selecting a group to be processed, 
wherein in th step (a), an inter frame is selected from the video stream group 
selected in the step (p-c). 
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Preferably, after the step (g), there are further included the steps of (h) 
checking whether the selected group is the last one, (i) if it is determined in the step 
(h) that the selected group is not the last one. selecting the next group, and 0) 
performing the steps (a) through (h) with respect to the next group selected in the 
step (i). 

Also, the predetermined algorithm is preferably a scene cut detection 
algorithm. 

Preferably, the video streams are moving picture expert group (MPEG)-7 
video streams. i 

Also, the predetermined histogram information is defined by a region intensity 
histogram (RIH) which indicates the intensity of a given video scene region. 

According to another aspect of the present invention, there is provided a 
digital video processing method comprising the step of defining the distribution of 
blocks whose motion compensation values are zero as motion activity, with respect 
to inter frames. 

To achieve the second object of the present invention, there is provided a 
computer readable medium having program codes executable by a computer to 
perform a digital video processing method comprising the steps of (a) selecting an 
inter frame from the video streams, (b) with respect to the selected inter frame, 
calculating the ratio of the number of blocks whose motion compensation values are 
zero to the total number of macroblocks, (c) quantizing the ratio into / levels, where / 
is a predetermined positive number, (d) checking whether the processed inter frame 
is the last one, (e) if it Is detemiined in the step (d) that the processed inter frame is 
not the fast one, selecting the next inter fl-ame and performing the steps (b) through 
(d), (f) if it is determined in the step (d) that the processed inter frame is the last one, 
calculating the ratio of the number of inter frames in the Ith level to the total number 
of inter flames In a given region, and (g) determining the ratio obtained in the step (f) 
as predetermined histogram information. 

To achieve the thinj object of the present invention, there is provided a digital 
video processing apparatus including grouping means for grouping video streams 
using a predetermined algorithm, selecting a group to be processed, and selecting 
and outputting each inter frame from the selected group in sequence, ratio 



calculating means for calculating the ratio of the number of blocks whose motion 
compensation values are zero to the total number of macroblocks, with respect to 
the selected inter frame, quantizing means for quantizing the ratio into / levels, 
where / is a predetermined positive number, and outputting level indication signals 
which indicate the respective levels, and a region intensity histogram (RIH) 
calculating means for calculating the ratio of the number of inter frames in the /th 
level to the total number of inter frames In a region, and outputting the calculation 
result as RIH information. 

To achieve the fourth object of the present invention, there is provided a 
digital video analyzing method comprising the step of calculating the difference 
between RIH^ and RIH2, which is denoted by Distance (RIH^, RIH^, as represented 
by the following Expression: 

Dis tan ce (RIHj , RIH^ ) = (RIH,- RIH 2/ M-'(RIH, , RIH^ ) 

wherein, of two video streams in a given video scene, RIH information of the first 
video stream is RIH information of the second video stream is RIH2, and M is a 
covariance matrix. 

To achieve the fifth object of the present invention, there is provided a method 
for searching for a desired digital video scene in compressed video streams 
including the step of (a) receiving video streams, (b) grouping the video streams into 
a plurality of groups using a predetermined algorithm, (c) selecting a group to be 
processed, (d) determining a region intensity histogram (RIH) of the selected group 
based on infomnation on motion compensation of inter frames in the selected group, 
(e) selecting a video scene to be searched, (f) retrieving RIH information of the 
selected video scene, and (g) searching regions having similar RIH information 
based on the retrieved RIH information, in each of the processed video streams. 

To achieve the sixth object of the present invention, there is provided an 
apparatus for searching for a desired digital video scene in compressed video 
streams including grouping means for grouping input video streams into a plurality of 
groups using a predetermined algorithm, motion compensation information retrieving 
means for retrieving infonnation on motion compensation of inter frames from a 
selected group and outputting processed video streams including the information as 



the RIH Information of the selected group, and searching means for retrieving RIH 
information of a selected group to be searched and searching for regions having 
similar RIH information based on the retrieved RIH information. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The above objects and advantages of the present Invention will become more 

apparent by describing in detail preferred embodiments thereof with reference to the 

attached drawings In which: 

FIG. 1 is a flow diagram showing a digital video processing method according 

to the present invention; and 

FIG. 2 is a block diagram of a digital video processing apparatus according to 

the present Invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Hereinafter, embodiments of the present invention will be described In detail 
with reference to the accompanying drawings. 

In the present Invention, in order to classify intensity of a video scene, motion 
Information is used. Referring to FIG. 1, the digital video processing method 
according to the present invention includes the step of inputting video streams, 
preferably, MPEG-2 video streams, (step 100). Next, the video streams are grouped 
using a scene cut detection algorithm (step 102). Then, a group of video streams to 
be processed Is selected (step 104). As known to one skilled in the art of MPEG, 
each of the video streams consists of a plurality of video frames. Video frames are 
classified as intra frames and Inter frames. In the field of MPEG technology, the 
intra frames are generally referred to as I frames, and the inter frames include P 
frames and B frames. For convenience' sake of explanation, let B frames be P 
frames, then, the video frames are classified as I frames and P frames, in other 
words, the selected group of video streams consist of I frames and P frames. 

Next, each P frame of the selected group of video streams Is selected In 
sequence (step 106). Then, In step 108, with respect to the selected P frame, the 
ratio (R) of the number of blocks whose motion compensation values are zero to the 
total number of macroblocks is calculated, as expressed in Expression (1): 



Number of NO MC blocks 
Total number of macroblocks 

wherein NOJMC blocks are blocks whose motion compensation values are zero, 
among macroblocks constituting the P frame. With respect to inter frames, the 
distribution of the blocks whose motion compensation values are zero is defined as 
motion activity. 

Next, the ratio R is quantized into one of five levels I,, Ij, Ig, I4 and I5. For 
example, the levels 1^, I2, 13. 14 and I5 refer to the cases where the ratios R are less 
than 5%, greater than or equal to 5% and less than 10%, greater than or equal to 
10% and less than 30%, greater than or equal to 30% and less than 50%, and 
greater than or equal to 50%. respectively. In such a manner, processing of each P 
frame is completed. 

Next, it is checked whether the processed P frame is the last one (step 1 12). 
If not. the next P frame Is selected (step 1 14) and the steps 106 through 1 10 are 
performed until the last P frame is processed. Then, supposing / is an integer from 1 
to 5, and the ratio of the number of P frames in the /th level ((,) to the total number of 
P frames in a region is denoted by P„ P, is calculated as follows: 

Number of P frames in I 

p = — '■ (2) 

' Total number of P frames in a region 

to determine a region intensity histogram (RIH) as follows: 

RIH = (Pi, P2, P3. P, , Ps) (3) 

where the region denotes a selected group (step 116). 

Then, it is checked whether the selected group is the last one (step 118). If 
not, the next group is selected (step 120) and the steps 106 through 1 16 are 
performed. In such a manner, the video processing for all groups is completed. 

According to the digital video processing method, the RIH information is 
obtained based on the motion compensation values of P frames. The RIH 
infomiation is a good indicator of the motion information of a video scene. Also, RIH 
infomnation is quite a good indicator of the intensity of the video scene. 
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The digital video processing method is programmable by a computer. Codes 
and code segments constituting a computer program can be easily derived by a 
computer programmer in the art. Also, the program is stored in computer readable 
media and is read and executed by the computer, thereby embodying the digital 
video processing method. The media include magnetic recording media, optical 
recording media, carrier wave media, and the like. 

Also, the digital video processing method can be implemented on a digital 
video processing apparatus. FIG. 2 is a block diagram of the digital video 
processing apparatus according to the present invention. Referring to FIG. 2, the 
digital video processing apparatus includes a grouping unit 200, a NO_MC block 
ratio calculating unit 202, a quantizing unit 204, an RIH calculating unit 206, and an 
RIH information storage unit 208. 

In the operation of the digital video processing apparatus, the grouping unit 
200 groups video streams using a scene cut detection algorithm, selects a group to 
be processed, and selects and outputs each P frame from the selected group in 
sequence. 

The NO_MC block ratio calculating unit 202 calculates the ratio of the number 
of blocks whose motion compensation values are zero to the total number of 
macroblocks, as defined in Expression (1) to output a ratio data R representing the 
calculated ratio. 

The quantizing unit 204 quantizes the ratio R into one of five levels I,, I2, U, U 
and Is to then output level indication signals I„ I2, 13, 14 and Ij, which indicate the 
respective levels. The respective levels 1^, I2. 13, U and I5 are for the cases where the 
ratios R are less than 5%, greater than or equal to 5% and less than 10%, greater 
than or equal to 10% and less than 30%, greater than or equal to 30% and less than 
50%, and greater than or equal to 50%. respectively. In such a manner, processing 
of each P frame is completed. 

If processing of the last P fi'ame is completed, the RIH calculating unit 206 
calculates the ratio of the number of P frames in the ilh level (/,) to the total number 
of P frames in a region, which is denoted by P;, as represented by Expression (2) to 
then output region intensity histogram (RIH) data, as represented by Expression (3). 



The RIH information storage unit 208 receives the R)H information (P„ P2, P3, 
P4 , P5) and stores the same therein. More preferably, the RIH information storage 
unit 208 receives video stream data, interleaves the RIH information with the video 
stream data and outputs RIH infomiation interleaved video streams. 

By the video streams processed by the digital video processing method 
shown in FIG. 1 and using the digital video processing apparatus shown in FIG. 2, 
the intensities of two arbitrary video streams can be compared and analyzed. 

According to a digital video analyzing method of the present invention, the 
difference between RIH1 and R/Hj, I.e., Distance (RIH^, RIH^), is calculated as 
represented by Expression (4): 

Distance(RIH,,RJH,)= (RIH- RIH,/ M-'(RIH,,RIH,) (4) 

wherein, of two video streams in a given video scene. RIH infonnation of the first 
video stream is RIH^, RIH infonnation of the second video stream is RiH^, and M is a 
covariance matrix. 

The Distance (RIMp RIH^ calculated by the above-described digital video 
analyzing method is properly indicative of the difference in intensity between two 
video streams. 

Using the video streams processed by the digital video processing method 
shown in FIG. 1 , which is implemented on the digital video processing apparatus 
shown in FIG. 2, region searching can be effectively performed by a digital video 
searching method according to the present invention. 

The digital video searching method according to the present invention is 
based on the assumption that the video streams are processed by the digital video 
processing method shown in FIG. 1 . In other words, it should be understood that 
the digital video searching method according to the present invention encompasses 
the digital video processing method which has been described with reference to FIG. 
1. 

According to the digital video searching method of the present invention, the 
RIH information of a selected video scene, that is. a video region, is first obtained. 

Next, with respect to the video streams processed by the digital video 
processing method shown In FIG. 1, regions having similar RIH information, that is. 
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video scenes having similar intensity, are searched for based on the obtained RIH 
information. 

Since the RIH information is detennined based on motion compensation 
values of P frames, it is properly indicative of motion information of the video scene. 
Also, the distribution of blocks whose motion compensation values with respect to 
inter frames are zero, accounts for motion activity quite well. Further, in a video 
program, in particular, in a sports program, the motion information is very closely 
related to the content of the pertinent video stream. The RIH information which Is 
based on motion information is quite a good indicator ofithe intensity of a video 
scene. In other words, the video streams of scenes having similar intensities can be 
effectively searched by searching for similar video scenes based on the RIH 
information. 

The above-described digital video searching method can be applied to multi- 
media push/pull application fields such as content based video search engines. 

Also, the digital video searching method Is programmable by a computer 
program. Codes and code segments constituting a computer program can be easily 
derived by a computer programmer in the art. Also, the program is stored in 
computer readable media and is readable and executable by the computer, thereby 
embodying the digital video searching method. The media include magnetic 
recording media, optical recording media, carrier wave media, and the like. 

Also, the digital video processing method can be embodied on a digital video 
searching apparatus (not shown). The digital video searching apparatus according 
to the present Invention includes a grouping unit, a motion compensation Infomiation 
retrieving unit, and a searching unit. 

In the operation of the digital video searching apparatus, the grouping unit 
groups input video streams into a plurality of groups using a scene cut detection 
algorithm. 

The motion compensation information retrieving unit extracts information on 
motion compensation of P frames from a selected group and outputs processed 
video streams including the information as the RIH information of the selected group. 
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The searching unit extracts RIH information of a selected group to be 
searched and searches for regions having similar RIH information based on the 
retrieved RIH infomriation. 

The aforementioned digital video searching apparatus can effectively search 
the video streams of scenes having similar intensities by searching for similar video 
scenes based on the RIH information. 

As described above, according to the present invention, the RIH infomiation 
is obtained based on the motion compensation values of inter frames, and the RIH 
information is a good indicator of motion information of a video scene. Also, since 
the RIH infomiation is quite a good indicator of intensity of the video scene, video 
streams having similar intensities can be effectively searched by searching for 
similar video scenes based on the RIH information obtained by a digital video 
processing method of the present invention. 



